New Directions in Online Learning: Boosting, Partial Information, and Non-Stationarity

Online learning, where a learning algorithm fits a model on-the-fly with streaming data, has become an important research area in machine learning. Batch learning, where the entire data set has to be available to the learning algorithm, is not always a suitable paradigm for the big data era. It is increasingly common in many practical situations, such as online ads prediction or control of self-driving cars, that data instances naturally arrive in a sequential manner. In these situations, researchers want to update their model in an online fashion. This dissertation pursues several topics at the frontier of online learning research. In Chapter 2 and Chapter 3, the journey starts with online boosting. Online boosting studies how to combine multiple online weak learners to get a stronger learner. Chapter 2 considers online multi-class classification problems. Chapter 3 focuses on the more challenging multi-label ranking problem where there are multiple correct labels and the learner outputs a ranking of labels based on their relevance. In both chapters, an optimal algorithm and an adaptive algorithm are proposed. The optimal algorithms require a minimal number of weak learners to attain the desired accuracy. The adaptive algorithms are practically more useful since they do not require a priori knowledge about the strength of weak learners and are more computationally efficient. The adaptive algorithms are not statistically optimal but they still come with reasonable performance guarantees. The empirical results on real data sets support the theoretical findings and the proposed boosting algorithms outperformed existing competitors on benchmark data sets. Chapter 4 considers the partial information setting, where the learner does not receive the true labels. Partial feedback is common in practice as obtaining complete feedback can be costly. The chapter revisits the boosting algorithms that are presented in Chapter 2 and Chapter 3 and extends them to work with partial information feedback. Despite the learner receiving much less information, comparable performance guarantees can be made. Later in Chapter 5 and Chapter 6, we move on to another interesting area in online learning called restless bandit problems. Unlike the classical (stochastic) multi-armed bandit problems where the reward distributions are unknown but stationary, in restless bandit problems the distributions can change over time. This extra layer of complexity allows us to study more complicated models, but the analysis becomes even more difficult. In restless bandit problems, it is assumed that each arm has a state that evolves according to an unknown Markov process, and the reward distribution depends on the arm's current state. This setting can be thought of as a sub-class of reinforcement learning and the partial observability inherent in this problem makes the analysis very challenging. The well known Thompson Sampling algorithm is analyzed and a Bayesian regret bound for it is derived. Chapter 5 considers the episodic case where the system periodically resets. Chapter 6 extends the analysis to the more challenging non-episodic (i.e., infinite time horizon) case. In both settings, Thompson Sampling algorithms (with slight modifications) enjoy sub-linear regret bounds, and the empirical results on simulated data support this fact. The experiments also suggest the possibility that the algorithm can be used in the frequentist setting even though the theoretical bounds are only shown for the Bayesian regret.PHDStatisticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/155110/1/yhjung_1.pd

Rotting infinitely many-armed bandits

We consider the infinitely many-armed bandit problem with rotting rewards, where the mean reward of an arm decreases at each pull of the arm according to an arbitrary trend with maximum rotting rate ϱ=o(1). We show that this learning problem has an Ω(max{ϱ1/3T,T−−√}) worst-case regret lower bound where T is the time horizon. We show that a matching upper bound O~(max{ϱ1/3T,T−−√}), up to a poly-logarithmic factor, can be achieved by an algorithm that uses a UCB index for each arm and a threshold value to decide whether to continue pulling an arm or remove the arm from further consideration, when the algorithm knows the value of the maximum rotting rate ϱ. We also show that an O~(max{ϱ1/3T,T3/4}) regret upper bound can be achieved by an algorithm that does not know the value of ϱ, by using an adaptive UCB index along with an adaptive threshold value

Contextual Linear Bandits under Noisy Features: Towards Bayesian Oracles

We study contextual linear bandit problems under uncertainty on features; they are noisy with missing entries. To address the challenges from the noise, we analyze Bayesian oracles given observed noisy features. Our Bayesian analysis finds that the optimal hypothesis can be far from the underlying realizability function, depending on noise characteristics, which is highly non-intuitive and does not occur for classical noiseless setups. This implies that classical approaches cannot guarantee a non-trivial regret bound. We thus propose an algorithm aiming at the Bayesian oracle from observed information under this model, achieving $\tilde{O}(d\sqrt{T})$ regret bound with respect to feature dimension $d$ and time horizon $T$. We demonstrate the proposed algorithm using synthetic and real-world datasets.Comment: 30 page

Effect of accumulated vs continuous exercise on excess post-exercise oxygen consumption

Background: A continuous aerobic exercise program is an effective method of improving calorie consumption on the metabolism of skeletal muscle. However, studies report that accumulated exercise of 30 minutes divided into three sessions of 10 minutes is as effective as one continuous exercise session for 30 minutes. As yet, no study has compared the excess post-exercise oxygen consumption associated with accumulated exercise and continuous exercise over these timeframes. Objective: The primary purpose of this study was to compare the excess post-exercise oxygen consumption associated with performing continuous exercise for 30 minutes and three sessions of accumulated exercise for 10 minutes at the same intensity of 60% VO2max. Method: Posters about the study were posted on the February 2019 Konkuk university homepage and bulletin board, and a total of 34 college students (males, n=18; females, n=16) volunteered to participate. Using a balanced repeated-measures crossover design, the subjects randomly took two exercises: continuous exercise (1 x 30 minutes) or accumulated exercise (3 x 10 minutes), and the washout period between the two exercises was a week. All exercises were performed using an ergometer at 60% maximal oxygen consumption. Oxygen consumption and heart rate were monitored and measured during exercise and after exercise. Lipid profile and lactate acid were measured at rest, exercise end, exercise end plus 30 minutes, and exercise end plus 60 minutes. IBM SPSS Statistics 23 was used to perform a paired t-test, and the statistically significant difference was set at <.05. Results: Excess post-exercise oxygen consumption parameters (e.g., total oxygen consumption, total calorie, and summation of heart rate) were higher in accumulated exercise than in continuous exercise (p<.05). No significant difference in the calorie during exercise between CEx and AEx (p = .140). No significant difference was observed in the lipid profile between accumulated exercise and continuous exercise (p>.05). No significant differences were observed at rest, exercise end plus 30 minutes, exercise end plus 60 minutes in lactic acid in the blood (p <.05). However, at exercise end, it was significantly higher in the accumulated exercise (p<.01). Conclusions: This study confirmed that after equalizing energy expenditure for continuous exercise and accumulated exercise in participants in their 20s, accumulated exercise results in higher excess post-exercise oxygen consumption than continuous exercise. The data suggests that accumulated exercise may be more effective in reducing body fat than continuous exercise for a given amount of energy expenditure. [Ethiop.J. Health Dev. 2020;34(Special issue-3):84-90] Keywords: Continuous exercise, accumulated exercise, excess post-exercise oxygen consumptio

A Case of Autoimmune Hemolytic Anemia Associated with an Ovarian Teratoma

Autoimmune hemolytic anemia associated with an ovarian teratoma is a very rare disease. However, treating teratoma is the only method to cure the hemolytic anemia, so it is necessary to include ovarian teratoma in the differential diagnosis of autoimmune hemolytic anemia. We report herein on a case of a young adult patient who had severe autoimmune hemolytic anemia that was induced by an ovarian teratoma. A 25-yr-old woman complained of general weakness and dizziness for 1 week. The hemoglobin level was 4.2 g/dL, and the direct and indirect antiglobulin tests were all positive. The abdominal computed tomography scan revealed a huge left ovarian mass, and this indicated a teratoma. She was refractory to corticosteroid therapy; however, after surgical resection of the ovarian mass, the hemoglobin level and the reticulocyte count were gradually normalized. The mass was well encapsulated and contained hair and teeth. She was diagnosed as having autoimmune hemolytic anemia associated with an ovarian teratoma. To the best of our knowledge, this is the first such a case to be reported in Korea